Virtualization detection

ABSTRACT

A baseline set of graph information associated with a predefined execution environment of a program may be obtained, the baseline set associated with a baseline time interval. The predefined execution environment includes predefined environment objects, at least a portion of which are dynamically mutatable. A second set of graph information associated with the predefined execution environment may be obtained, the second set associated with a second time interval that is later in time than the baseline time interval. The baseline set and the second set may be compared to detect virtualization activity associated with a dynamic runtime.

BACKGROUND

Developers and users of applications such as web applications may relyon execution environments to ensure that their applications execute inaccordance with user expectations. Dynamic execution environments suchas JAVASCRIPT execution environments may define reflection interfacesthat allow applications to dynamically introspect and modify core systemobjects. However, this may lead to virtualization events, in which, forexample, a virtualizing enclosing environment may present an applicationwith an ostensibly unmodified execution environment (e.g., a JAVASCRIPTruntime); however, the execution environment may have been modified bythe virtualizing enclosing environment, which may cause the applicationto experience unexpected results (or to potentially experienceundesirable interference from the virtualizing enclosing environment).

SUMMARY

According to one general aspect, a system may include a device thatincludes at least one processor. The device may include a virtualizationdetection engine that includes instructions tangibly embodied on acomputer readable storage medium for execution by the processor. Thevirtualization detection engine may include an environment statedetermination component configured to obtain a baseline set of graphinformation associated with a predefined execution environment of aprogram. The baseline set may be associated with a baseline timeinterval, and the predefined execution environment may includepredefined environment objects, at least a portion of which may bedynamically mutatable. The environment state determination component mayobtain a second set of graph information associated with the predefinedexecution environment, the second set associated with a second timeinterval that is later in time than the baseline time interval. Anenvironment state comparison component may be configured to compare thebaseline set and the second set to detect virtualization activityassociated with a dynamic runtime.

According to another aspect, a baseline set of graph informationassociated with a predefined JAVASCRIPT execution environment of a webbrowser may be obtained, the baseline set indicating a baseline state ofthe predefined JAVASCRIPT execution environment at a baseline timeinterval. A second set of graph information associated with thepredefined JAVASCRIPT execution environment may be obtained, the secondset indicating a second state of the predefined JAVASCRIPT executionenvironment at a second time interval that is later in time than thebaseline time interval. The baseline set and the second set may becompared to detect virtualization activity associated with a dynamicruntime.

According to another aspect, a computer program product tangiblyembodied on a computer-readable storage medium may include executablecode that may cause at least one data processing apparatus to obtain,via a device processor, a baseline set of state information associatedwith predefined execution environment objects that are included in apredefined execution environment of a program, at least a portion of thepredefined execution environment objects being dynamically mutatable,the baseline set of state information indicating a baseline state of thepredefined execution environment at a baseline time interval. Further,the data processing apparatus may obtain a second set of stateinformation associated with the predefined execution environmentobjects, the second set of state information indicating a second stateof the predefined execution environment at a second time interval thatis later in time than the baseline time interval. Further, the dataprocessing apparatus may initiate a comparison of the baseline set ofstate information and the second set of state information to determinevirtualization activity associated with a dynamic runtime.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. The details ofone or more implementations are set forth in the accompanying drawingsand the description below. Other features will be apparent from thedescription and drawings, and from the claims.

DRAWINGS

FIG. 1 is a block diagram illustrating of an example system fordetecting virtualization in dynamic runtime environments.

FIGS. 2 a-2 b are block diagrams of example applications and executionenvironments.

FIGS. 3 a-3 e are a flowchart illustrating example operations of thesystem of FIG. 1.

FIGS. 4 a-4 c are a flowchart illustrating example operations of thesystem of FIG. 1.

FIG. 5 is a flowchart illustrating example operations of the system ofFIG. 1.

DETAILED DESCRIPTION

I. Introduction

Execution environments are used by applications such as web applicationsas a basis for expected execution performance by users of theapplications. For example, a JAVASCRIPT runtime has been widely used asan execution substrate for web applications. This runtime may definereflection interfaces that allow applications to dynamically introspectand modify core system objects. However, the runtime may provide no wayfor a JAVASCRIPT library to definitively access the native, unmodifiedversions of core runtime objects. Thus, a naive web application may bevulnerable to virtualization attacks, if a malicious enclosingenvironment (e.g., a page) presents the application with an ostensiblyunmodified JAVASCRIPT runtime. However, the runtime may have actuallybeen specially crafted by the malicious environment, and the runtime mayallow such an attacker to access the application's inputs, tamper withthe application's local storage, etc. It is also possible that anon-malicious enclosing environment may provide an application with amodified runtime environment that may cause unintended unreliableresults.

For example, benevolent programs may also use virtualizations. Forexample, a MUGSHOT logging and replay tool may virtualize a web page'sruntime to provide a useful, non-malicious service (e.g., the logging ofapplication behavior for subsequent diagnosis of software bugs), asdiscussed in J. Mickens, et al., “MUGSHOT: Deterministic Capture andReplay for JAVASCRIPT Applications,” In Proceedings of the 7^(th) USENIXSymposium on Networked Systems Design and Implementation (NSDI'10), SanJose, Calif., Apr. 28-30, 2010. Even though this type of virtualizationmay be considered non-malicious, an application user may wish to avoidrunning applications in such a virtualized environment.

In this context, a “dynamic execution environment” may refer to anexecution environment in which an application may be able to redefinesome or all of the predefined objects and object references associatedwith the dynamic execution environment (e.g., functions such as theJAVASCRIPT “date” function).

Recent research results have described example techniques by whichapplications may be able to detect virtualized environments, e.g., bylooking for behavioral quirks that are unique to a particularvirtualization environment, but which do not exist in non-virtualizedenvironments. These example approaches may assume that the runtimeenvironment itself provides no anti-virtualization support toapplications.

Example techniques discussed herein may involve an example service(e.g., a browser service) that uses properties of an application'sobject graph (e.g., other than behavioral properties) to determinewhether a runtime (e.g., a JAVASCRIPT runtime) has been virtualized. Forexample, a secure web application may invoke this browser service todetermine whether the application has been launched within a virtualizedenvironment. If so, the application may refuse to execute, or may takeother security measures. Such example techniques may be used, forexample, in program execution environments that support dynamicredefinition of runtime objects (e.g., JAVASCRIPT, PYTHON interpreter,RUBY interpreter).

In this context, “virtualization” of an environment, or a “virtualized”environment may refer to an environment that are “counterfeit” or “fake”environments that are presented to applications in lieu of an actualenvironment. For example, an application running in a “virtualized”execution environment may execute using virtualized executionenvironment objects (e.g., functions, variables, constants) in the sameway that the application would run in a non-virtualized executionenvironment, without realizing that the environment objects may havebeen modified by the “virtualized” execution environment. For example,an application may be configured to render a message “Hello” on adisplay device. Thus, the application may access one or more executionenvironment objects for rendering the message “Hello” on the displaydevice. However, if the application is instead executing in a“virtualized” environment in which the execution environment objectshave been modified to instead send “I want to break up with you, now” toa display of user's loved one, a user of the application may experienceundesired results.

As another example, a user may request that a current date may bedisplayed, by calling an execution environment object such as a “date”function that may be configured to access a system date to provide thatvalue for the user. However, a “virtualizing” environment may insteadintercept the user's request via the call to the “date” function, andredefine the function to instead return a different result that isunexpected by the user.

As another example, an attacker may virtualize a victim environment'sinterface to local storage, allowing the attacker to arbitrarily inspectand modify persistent client-side state. For example, the attacker mayalso inject a remote debugging engine into a victim environment,allowing the attacker to examine the environment's JAVASCRIPT state inreal time, as discussed in J. Mickens, “Rivet: Browser-agnostic RemoteDebugging for Web Applications,” In Proceedings of USENIX AnnualTechnical Conference (ATC'12), Boston, Mass., Jun. 13-15, 2012. Asanother example, an attacker may virtualize an entire JAVASCRIPT runtimeand mediate substantially all of the victim environment's interactionswith the outside world (e.g., the network, local storage, the screen,user input devices, and other frames on the browser).

In accordance with example techniques discussed herein, applications maydetect virtualization attacks in dynamic execution platforms byleveraging predefined platform functions that reveal informationassociated with an application's object graph. Using these functions,applications may perform graph comparisons between a baseline, trusted(e.g., “known-good”) object graph, and the object graph currentlyassociated with the application. Differences between the baseline graphand the current graph may indicate a virtualization attack.

For example, the platform may provide a predefined function, which maybe referred to herein as a “detectVirt( ) function,” which may directlycompare a baseline snapshot of the object graph to a snapshot of thecurrent version of that graph.

For example, the platform may not provide a detectVirt( ) function, or auser may desire other means for detecting virtualizations. In accordancewith example techniques discussed herein, if the platform provides otherfunctions that reveal information associated with the object graph, andthe code that defines such functions, then the application may firstvalidate those functions, for example, by comparing their outputs to“known-good” expected outputs. The application may then use thosefunctions to determine the structure of the current object graph, andcompare that structure to a “known-good” structure.

II. Example Operating Environment

Features discussed herein are provided as example embodiments that maybe implemented in many different ways that may be understood by one ofskill in the art of data processing, without departing from the spiritof the discussion herein. Such features are to be construed only asexample embodiment features, and are not intended to be construed aslimiting to only those detailed descriptions.

As further discussed herein, FIG. 1 is a block diagram of a system 100for detecting virtualization in dynamic runtime environments. As shownin FIG. 1, a system 100 may include a device 101 that includes avirtualization detection engine 102 that includes an environment statedetermination component 104 that may be configured to obtain a baselineset 106 of graph information associated with a predefined executionenvironment 108 of a program 110. The baseline set 106 is associatedwith a baseline time interval. For example, the baseline set 106 mayindicate an initial state of the predefined execution environment 108(e.g., at an initialization time of execution).

For example, the predefined execution environment 108 may include aJAVASCRIPT execution environment, and the program 110 may include abrowser (e.g., INTERNET EXPLORER, MOZILLA FIREFOX, NETSCAPE, GOOGLECHROME). The predefined execution environment 108 includes predefinedenvironment objects 114, at least a portion of which are dynamicallymutatable. The environment state determination component 104 may obtaina second set 116 of graph information associated with the predefinedexecution environment 108, the second set 116 associated with a secondtime interval that is later in time than the baseline time interval.

Dynamic execution environments (e.g., JAVASCRIPT, RUBY, PYTHON) involvepredefined application runtimes. These predefined runtimes may includeservices that many programs may use frequently. For example, theseservices may include methods for accessing the current date, or sendinginformation to a remote computer. Such predefined execution environmentsrepresent respective graphs. For example each node in the graph mayinclude a predefined object or method, and each edge in the graph mayinclude a reference that links two objects/methods. For example, incurrent web browsers, a HyperText Markup Language (HTML) structure of aweb page may be represented in JAVASCRIPT using a data structure such asa Document Object Model (DOM) tree. For example, each element in the DOMtree is a JAVASCRIPT object. For example, if two HTML tags have aparent-child relationship, they may also have a parent-childrelationship in the DOM tree (and thus an edge in the JAVASCRIPT objectgraph discussed above).

For example, virtualization activity may modify (e.g., via tamperingactivity) a runtime's predefined object graph. For example, in anexample predefined JAVASCRIPT runtime that may be used by a browser, apredefined window object may include a reference to a predefinedfunction named Date( ). Thus, a non-virtualized runtime for a JAVASCRIPTapplication may include an object representing the browser-suppliedversion of the window object, and may include an object representing thebrowser-supplied version of the Date( )method, and there may exist alink in the object graph between those two entities. In a virtualizationevent, a virtualizing entity (e.g., an attacker) might replace thepredefined window object's link to the predefined Date( ) method,causing that link to point to a new, modified (e.g., malicious) versionof the Date( ) method that was defined by the virtualizing entity, andwhich, for example, may returns an incorrect date to the application, ormay log application invocations of Date( ) and send those results to aremote computer.

Virtualization events such as virtualization attacks may result in achange to the object graph that represents the application's runtime.Thus, a comparison of the predefined object graph with the graphrepresenting a virtualized runtime may indicate that the two graphs aredifferent, i.e., they may have a different set of nodes and/or adifferent set of edges.

Example techniques discussed herein may perform such a graph comparison.For example, a web browser (or more generally, a dynamic executionenvironment such as a PYTHON interpreter) may define a method (e.g., afunction) which applications may invoke to determine whether thepredefined runtime has been virtualized. The method, or predefinedfunction, which may be referred to as detectVirt( ) for convenience, mayoperate as discussed further below.

As an example, after a browser generates a new execution context for anapplication (but before the new application is started), the browser maycollect graph information for all of the predefined objects; forexample, the browser may perform a breadth-first traversal of the objectgraph for each global variable, updating a list which represents theorder in which various objects are encountered during the traversal. Forexample, the browser stores the graph information in a storage areawhich is inaccessible to the application or a virtualization attacker.Later, when the application invokes the function detectVirt( ) thebrowser may collect graph information for the current object graph, andmay returns “true” if that information differs from the original datacollected, and “false” otherwise. For example, the browser may perform abreadth-first traversal of the execution environment's current graph,checking whether the same set of references are encountered in the sameorder as they were during the initial graph traversal. If detectVirt( )informs the application that the runtime has been virtualized, theapplication may terminate execution, attempt to warn the user that theruntime has been virtualized, or take some other action to prevent thevirtualized environment from influencing the application's execution inan undesired way.

For example, a JAVASCRIPT environment may include a set of globalvariables in the JAVASCRIPT namespace. Each global variable mayreference zero or more additional variables. In turn, each of thosevariables may reference zero or more additional variables. Byrecursively traversing the objects reachable from the global variables,a map of the object graph may be generated that represents the objectsin the JAVASCRIPT runtime. For example, a “snapshot” of the graph may begenerated, based on a breadth-first traversal (BFS) algorithm, using theset of global variables as the initial entries in the BFS queue. Nodesare then visited as the graph is traversed. Whenever a node is visited,a reference (e.g., a pointer) to that node may be stored in an initiallyempty list (e.g., if a node is a primitive value type such as a number,that primitive value may be stored in the list instead of a reference).When all of the objects have been visited, the list of references andprimitive values represents a “snapshot” of the graph. For example, thesnapshot may represent a log of the data encountered during the graphtraversal, in the order that in which the data was encountered.

The detectVirt( ) function itself is not virtualizable. In other words,a dynamic execution environment is characterized by an ability for anapplication to redefine some or all of the predefined objects and objectreferences. Applications may not redefine the detectVirt( ) function,nor may they redefine the names (i.e., the object references) that thesystem predefines to provide access to detectVirt( ). For example, thismay thus prevent a virtualization attacker from redefining detectVirt( )to remove its security checks. One skilled in the art of data processingwill understand that some aspects of a dynamic runtime may not bechangeable, while other aspects may be changeable.

According to an example embodiment, the virtualization detection engine102, or one or more portions thereof, may include executableinstructions that may be stored on a tangible computer-readable storagemedium, as discussed below. According to an example embodiment, thecomputer-readable storage medium may include any number of storagedevices, and any number of storage media types, including distributeddevices.

For example, an entity repository 120 may include one or more databases,and may be accessed via a database interface component 122. One skilledin the art of data processing will appreciate that there are manytechniques for storing repository information discussed herein, such asvarious types of database configurations (e.g., relational databases,hierarchical databases, distributed databases) and non-databaseconfigurations.

According to an example embodiment, the virtualization detection engine102 may include a memory 124 that may store the graph information. Inthis context, a “memory” may include a single memory device or multiplememory devices configured to store data and/or instructions. Further,the memory 124 may span multiple distributed storage devices.

According to an example embodiment, a user interface component 126 maymanage communications between a user 128 and the virtualizationdetection engine 102. The user 128 may be associated with a receivingdevice 130 that may be associated with a display 132 and otherinput/output devices. For example, the display 132 may be configured tocommunicate with the receiving device 130, via internal device buscommunications, or via at least one network connection.

According to example embodiments, the display 132 may be implemented asa flat screen display, a print form of display, a two-dimensionaldisplay, a three-dimensional display, a static display, a movingdisplay, sensory displays such as tactile output, audio output, and anyother form of output for communicating with a user (e.g., the user 128).

According to an example embodiment, the virtualization detection engine102 may include a network communication component 134 that may managenetwork communication between the virtualization detection engine 102and other entities that may communicate with the virtualizationdetection engine 102 via at least one network 136. For example, thenetwork 136 may include at least one of the Internet, at least onewireless network, or at least one wired network. For example, thenetwork 136 may include a cellular network, a radio network, or any typeof network that may support transmission of data for the virtualizationdetection engine 102. For example, the network communication component134 may manage network communications between the virtualizationdetection engine 102 and the receiving device 130. For example, thenetwork communication component 134 may manage network communicationbetween the user interface component 126 and the receiving device 130.

As shown in FIG. 1, the virtualization detection engine 102 may be incommunication with a device processor 138. In this context, a“processor” may include a single processor or multiple processorsconfigured to process instructions associated with a processing system.A processor may thus include one or more processors processinginstructions in parallel and/or in a distributed manner. Although thedevice processor 138 is depicted as external to the virtualizationdetection engine 102 in FIG. 1, one skilled in the art of dataprocessing will appreciate that the device processor 138 may beimplemented as a single component, and/or as distributed units which maybe located internally or externally to the virtualization detectionengine 102, and/or any of its elements.

For example, the system 100 may include one or more processors 138. Forexample, the system 100 may include at least one tangiblecomputer-readable storage medium storing instructions executable by theone or more processors, the executable instructions configured to causeat least one data processing apparatus to perform operations associatedwith various example components included in the system 100, as discussedherein. For example, the one or more processors 138 may be included inthe at least one data processing apparatus. One skilled in the art ofdata processing will understand that there are many configurations ofprocessors and data processing apparatuses that may be configured inaccordance with the discussion herein, without departing from the spiritof such discussion.

In this context, a “component” may refer to instructions or hardwarethat may be configured to perform certain operations. Such instructionsmay be included within component groups of instructions, or may bedistributed over more than one group. For example, some instructionsassociated with operations of a first component may be included in agroup of instructions associated with operations of a second component(or more components).

An environment state comparison component 140 may be configured tocompare the baseline set 106 and the second set 116 to detectvirtualization activity associated with a dynamic runtime.

For example, comparing two graphs for equivalence may involve obtaining“snapshots” of the two graphs. For example, if the snapshot lists havedifferent sizes, then the graphs may be considered unequal (i.e., amodifying entity has modified the runtime represented by the secondgraph). If the snapshots have the same size (or substantially the samesize), then it is possible to iterate through the snapshot lists,comparing the first element in both lists, and then the second elementin both lists, and so on. For example, if, at any point, the two itemsare not the same (or substantially the same), the two graphs may bedetermined as not being equivalent. If the entirety of both lists areiterated through and it is determined that all of the elements (orsubstantially all of the elements) are equal, then the two graphs may bedetermined as being equivalent (e.g., given a deterministic searchalgorithm and two isomorphic graphs, the search may visit the samevertices in each graph in the same order).

For example, the predefined environment objects 114 that are dynamicallymutatable may include one or more of environment variables 142,environment functions 144, environment references 146, environmentconstant values 148, or environment state values 149.

For example, the predefined execution environment 108 of the program 110may include one or more of a predefined JAVASCRIPT execution environmentof a web browser, a predefined RUBY execution environment, or apredefined PYTHON execution environment.

For example, second set 116 of graph information may be obtained viainvocation of a first non-virtualizable function 119 that is defined bythe predefined execution environment 108.

For example, a supplemental graph component 150 may be configured toobtain a first supplemental set 152 of graph information associated witha program graph that is induced based on predefined program objects ofthe program 110, and predefined relationships between the predefinedprogram objects, at the baseline time interval.

For example, the supplemental graph component 150 may be configured toobtain a second supplemental set 156 of graph information associatedwith the program graph, at the second time interval.

For example, the first supplemental set 152 may be added to the baselineset 106 of graph information via a second non-virtualizable function 158invoked by the program 110, the second non-virtualizable function 158defined by the predefined execution environment 108.

For example, the second supplemental set 156 may be added to the secondset 116 of graph information via the second non-virtualizable function158.

For example, the environment state comparison component 140 may beconfigured to compare the baseline set 106 and the second set 116 basedon comparing the baseline set 106 that includes the first supplementalset 152 and the second set 116 that includes the second supplemental set156, to detect the virtualization activity associated with the dynamicruntime.

For example, an application may add, remove, or otherwise modify theparts of the object graph that correspond to application-defined data.For example, when applications redefine their own data, this may notrepresent a virtualization attack, and detectVirt( ) need not checkthose parts of the object graph for modification. In other words, in a“minimal” implementation, detectVirt( ) may check the predefined objectgraph that is provided by the dynamic runtime (e.g., the web browser,the PYTHON interpreter, etc.). However, in accordance with anotheraspect, detectVirt( ) may check portions of the application-definedgraph for inappropriate (or unintended) modification. For example, thedynamic runtime may expose a second anti-virtualization function (ormethod), which may be referred to herein addRootToProtectedGraph(obj).By invoking this function, the application may inform the dynamicruntime (e.g., the web browser) to add the object graph reachable from“obj” to the larger graph that detectVirt( ) checks when it isdetermining potential virtualization activity (e.g., mutation activity).For example, it may be desirable that the functions detectVirt( )addRootToProtectedGraph( ) may not be virtualizable.

For example, the baseline set 106 of graph information may includebaseline state information associated with the predefined executionenvironment objects 114 and relationships between the predefinedexecution environment objects 114, at the baseline time interval. Forexample, a graph associated with the baseline set 106 of graphinformation is induced based on graph nodes representing the predefinedexecution environment objects 114 and graph edges representing therelationships between the predefined execution environment objects 114.

For example, the second set 116 of graph information may include secondstate information associated with the predefined execution environmentobjects 114 and the relationships between the predefined executionenvironment objects 114, at the second time interval.

For example, the environment state determination component 104 may beconfigured to obtain the baseline set 106 of graph information based onrecursively first traversing nodes representing predefined executionenvironment objects 114 that are reachable via references from globalvariables in the predefined execution environment 108, at the baselinetime interval, and storing values associated with first traversalvisited nodes in a first list 162.

For example, the environment state determination component 104 may beconfigured to obtain the second set 116 of graph information based onrecursively second traversing the nodes representing the predefinedexecution environment objects 114 that are reachable via references fromglobal variables in the predefined execution environment 108, at thesecond time interval, and storing values associated with secondtraversal visited nodes in a second list 164.

For example, the environment state comparison component 140 may beconfigured to compare the baseline set 106 and the second set 116 basedon one or more of comparing a size of the first list 162 with a size ofthe second list 164, or comparing a value of each element of the firstlist 162 to a value of a corresponding element of the second list 164.

For example, a graph caching component 166 may be configured to initiatestorage of the baseline set 106 of graph information in a privatestorage location 168 that is privately available to the predefinedexecution environment 108, wherein the baseline set 106 of graphinformation is available for use by the predefined execution environment108, in lieu of at least a portion of the predefined environment objects114 that are dynamically mutatable.

For example, the environment state determination component 104 may beconfigured to obtain a third set 170 of graph information associatedwith the predefined execution environment 108, at a third time intervalthat is later in time than the second time interval, the third set 170obtained via invocation of the first non-virtualizable function 119.

For example, the environment state comparison component 140 may beconfigured to compare the baseline set 106 and the third set 170 todetect virtualization activity associated with the dynamic runtime.

For example, an environment function validation component 172 may beconfigured to initiate validation of current output of at least onefunction that is defined by the predefined execution environment 108.For example, the baseline set 106 of graph information may be obtainedvia invocation of the at least one function after validation of output,as discussed further herein. For example, the second set 116 of graphinformation may be obtained via invocation of the at least one functionafter validation of output.

FIGS. 2 a-2 b are block diagrams of example applications and executionenvironments. As shown in FIG. 2 a, a web page 202 may use a browserinterface 204 to invoke services of a web browser 206. For example, theweb page 202 may use an application programming interface (API) of thebrowser interface 204 to invoke the services. As shown in FIG. 2 b, theweb page 202 may use a virtualization layer 208 to invoke the servicesof the web browser 206. For example, the virtualization layer 208 mayuse the browser interface 204 to access the services of the web browser206, and may modify actions and results requested by the web page 202(e.g., as a “fake” or “counterfeit” version of the browser interface204).

For example, JAVASCRIPT is a dynamically-typed, object-orientedscripting language. The core language defines basic data types such asstrings and regular expressions; recently, web browsers may extend thiscore with a DOM interface which may expose browser resources such ascookies, HyperText Transfer Protocol (HTTP) connections, andHTML/Cascading Style Sheets (CSS) state. The DOM interface also providesmechanisms for collecting graphical user interface (GUI) inputs,manipulating graphics, spawning new pages, and sending messages topreexisting client-side pages.

For example, the JAVASCRIPT language defines a substantially permissivereflection model. For example, each JAVASCRIPT object may represent adictionary that maps property names to property values. Usingintrospection APIs such as the for-in iteration statement and theObject.getOwnPropertyNames( ) call, an application may discover anobject's properties.

By default, object properties are mutable, and dynamic objectmodification is a JAVASCRIPT attribute. To limit such activity, anapplication may associate an object property with an access descriptor.A descriptor may mark its property as read-only, or associate theproperty with getter/setter functions that interpose on all accesses tothat property and perform context-sensitive access controls. However,such descriptors may be rarely used by the predefined JAVASCRIPT objectsin the core runtime, resulting in their properties being potentiallyvulnerable to virtualization by unscrupulous code. Furthermore,attackers may elude descriptor checks altogether by virtualizing anentire sensitive object instead of selectively replacing individualproperties on that object.

Further, JAVASCRIPT is an object-oriented language, but unlike JAVA orC#, JAVASCRIPT supports inheritance using dynamic prototypes instead ofstatic classes. Each JAVASCRIPT object has a special property called_(——)proto_(——) that points to another JAVASCRIPT object. That object,referred to as the prototype, represents an exemplar for all instancesof a class. Each instance inherits the properties defined by itsprototype; assigning to such a property on an instance object overwritesthe property definition for that instance. Since the prototype object isa regular JAVASCRIPT object, its properties may also be modified atruntime. Thus, the instance properties of a particular class may alsochange at runtime.

Code may also dynamically modify the class of an object by assigning anew value to its _(——)proto_(——) property. For a constructor function C,the property C.prototype is the default _(——)proto_(——) value forobjects created via new CO.

JAVASCRIPT functions are first-class objects, as that they may defineproperties similar to any other object. They also implement the methodsdefined by Function.prototype.

The predefined Function.prototype.toString( ) method returns a stringthat represents the source code for a function. If the function isdefined by the application, the string represents JAVASCRIPT sourcecode. However, if the function is a built-in, i.e., implemented by thebrowser in C++, Function.toString( ) returns “function funcName(){[native code]}”. For example, Function.prototype.toString( ) is anative method, as are the methods belonging to regular expressions andDOM objects.

In JAVASCRIPT, an iframe may represent an isolated JAVASCRIPT executioncontext. Each iframe may be single-threaded, containing an event loopthat executes application-defined event handlers for GUI activity, timercallbacks, the reception of network data, etc. A frame's origin mayinclude the protocol, hostname, and port used to fetch the frame's HTML.The same-origin policy prevents two frames with different origins fromaccessing each other's JAVASCRIPT state. However, two frames withdifferent origins may communicate using the postMessage( )JAVASCRIPTfunction. This function allows frames to exchange asynchronous,immutable strings with each other. Iframes provide substantial isolationbetween web applications.

Iframes may provide substantial isolation between web applications.However, within a given frame, JAVASCRIPT provides few mechanisms forcode to isolate itself from other code in that frame. Besides theproperty descriptors mentioned above, JAVASCRIPT provides closures,which are functions that lexically bind to the variables in theirenclosing stack frame. Algorithm 1 below illustrates an example of aclosure.

Algorithm 1 Closure Example function closureGenerator (x){ var y = 12;var closure = function ( ){ return x + y; }; return closure ; } var c =closureGenerator (30); alert (c ( )); // Displays “42”

As shown in Algorithm 1, when closureGenerator( ) is invoked, the innerfunction closure( ) binds to the variables x and y that reside in theactivation record for closureGenerator( ). When closureGenerator( )returns and the activation record pops off the stack, closure( )maintains a reference to the record. This allows closure( ) to lateraccess x and y.

Closures do not expose their bound variables to external code. Thus,closures may be used as information hiding mechanisms. For example, acaller may place sensitive data inside a closure, and pass the closureto untrusted code. The untrusted code will only be able to manipulatethe hidden variables by invoking the trusted closure function.

Virtualization attacks may subvert both iframe isolation and closureisolation. For example, by virtualizing DOM metadata, an attacker mayvirtualize the iframe interface, deluding a victim application intothinking that it has strong iframe isolation even when it does not.Closures are also vulnerable to virtualization attacks, since any globalvariable that they access may be a virtualized one.

The JAVASCRIPT runtime defines interfaces that allow an application tointeract with the outside world. For example, the JAVASCRIPT Date( )function returns the current time, and the XMLHttpRequest class allowsan application to fetch HTTP data from remote servers. The runtimeexposes local storage via the cookie interface (which is may be limitedto 4 KB of data), and the DOM storage interface (which provides eachorigin with key-value storage). The runtime also provides the DOM tree,which may include a JAVASCRIPT representation of a page's HTMLstructure. By manipulating the DOM tree, a program may dynamicallychange the structure and presentation of its visual content. Forexample, if these interfaces are virtualized, an attacker may returnfalse timestamps to a victim page, tamper with a victim's networktraffic, arbitrarily modify the victim's visual representation, etc.

The detectVirt( ) function discussed above may be implemented by thedynamic execution platform. However, applications may find themselvesrunning on dynamic execution platforms that do not support detectVirt(). These applications may still wish to detect virtualized environments.To do so, these applications may, for example, leverage other predefinedruntime functions that provide information about the current state ofthe application's object graph. For example, the application mayvalidate these functions by checking what the predefined runtimefunctions output when presented with a set of inputs whose outputs fromnon-virtualized functions are known in advance. For example, once thefunctions are validated, the application may use the functions tovalidate other parts of the object graph.

Example techniques discussed herein may leverage at least two built-inJAVASCRIPT functions to detect virtualized environments:Function.prototype.toString( ) and Object.prototype.toString( ). Forexample, Function.prototype.toString( ) allows a JAVASCRIPT program toaccess the source code of functions (unless the function is a built-infunction implemented by the browser in C++, in which caseFunction.prototype.toString( ) returns “function f( ){[native code]}”.When passed a native object, Object.prototype.toString( ) returns theclass of the object. For example, when passed the document object, itreturns “[object HTMLDocument]”; when passed a regular expression, itreturns “[object Reg-Exp]”.

To test for virtualized environments, an example technique may first runa suite of validity tests on Function.prototype.toString( ). If thosetests pass, Function.prototype.toString( ) may be trusted to report thesource code for an application-defined method, or the name and thenative code nature of a native function. For example,Function.prototype.toString( ) may then be used to validateObject.prototype.toString( ). Once Object.prototype.toString( ) isvalidated, it may be used to validate that ostensibly native runtimeobjects are actually native in origin. For example,Object.prototype.toString( ) may then be used to validate DOM objectssuch as documents.

For example, once core global methods and objects are validated,references to them may be cached in a dictionary. The application maylater use these references to interact with safe, known-good versions ofthe core runtime objects. For example, the validated XML-HttpRequestobject may then be used to synchronously fetch and eval( ) the secureapplication's JAVASCRIPT code.

For example, Function.prototype.toString( ) may be used to validatenative functions. For example, Algorithm 2 below illustrates an exampleof an attack.

Algorithm 2 Attack Example Attack 0 var Date = function( ){ returnmaliciousValue( ); };

As shown in Algorithm 2, Attack 0 represents a virtualization attack inwhich the malicious code overwrites a native function such as Date( )with attacker-controlled code. For example, Algorithm 3 illustrates anexample of an attack detection.

Algorithm 3 Detection Example Detection 0 if(Date.toString( ) !=“function Date( ){ [native code] }”){ throw “Attack detected”; }

As shown in Algorithm 3, Detection 0 represents a detection of suchattacks. As shown in Algorithm 3, the native method Date.toString( ) maybe invoked, to return the source code for the now-virtualized functioninstead of the expected source code of “function Date( ){[nativecode]}”.

One skilled in the art of data processing that many other types ofdetection may be utilized, without departing from the spirit of thediscussion herein.

JAVASCRIPT is single-threaded within a particular frame. After a frame'sinitial JAVASCRIPT code runs, the browser may fire application-definedevent handlers, but it may execute one handler at any given time. Thus,although the DOM tree may be validated at initialization time, anattacker may subsequently virtualize the DOM tree between invocations ofa secure application's event handlers. Therefore, validated copies ofFunction.prototype.toString( ) and Object.prototype.toString( ) may becached, so that, for example, DOM objects may continue to be validatedon the behalf of the secure application. However, if a secureapplication is concerned with the visual layout or GUI inputs of a page,then whenever a DOM-manipulating handler loads, it may re-validate theentire DOM tree before updating it. Such revalidation may be expensivefor visually complex sites. Therefore, DOM-intensive secure applicationsmay use example techniques discussed herein to validate a minimal amountof state for creating a virgin iframe via JAVASCRIPT; the applicationmay then spawn a full version of the application inside this virgin,known-clean frame. The full version of the page may use frame-bustingcode with confidence because example techniques discussed herein maydetermine that the new frame is non-virtualized. For example, a librarymay explicitly define a bootstrap( ) function to perform such validationfollowed by a frame-busting page launch.

Re-validation issues may not be a concern for libraries that do notmanipulate the DOM tree. For example, a cryptographic library or anRPC-over-AJAX library may not habitually re-validate cached versions ofMath.random( ) or XMLHttpRequest, as the virtualization attacker may notsuccessfully rewrite the obfuscated code, and thus may not puncture theclosure in which example techniques herein may stores cached referencesto native runtime objects. Under this assumption, other obfuscatedapplications may also protect closured references to native objects.

For example, browsers may define a detectVirt( ) function to assistsecurity-sensitive applications. At load time, a secure application maycall detectVirt( ) The browser may inspect the global runtime objectssuch as Date( ) XMLHttpRequest, and Math.random( ); for each variable,the browser may inspect the object graph rooted at that variable,checking for isomorphism between that graph and the one that is rootedby the browser's canonical version of that global variable. For example,when the browser itself loads, it may perform a breadth-first traversalof the object graph for each global variable, compiling a list of thereferences encountered during the traversal. Later, when the browserchecks a global variable for virtualization, it may performs abreadth-first traversal of the global object's current graph, checkingwhether the same set of references are encountered in the same order.

If the two graphs are isomorphic, i.e., they define the samerelationships between prototype objects and object properties, then theassociated global variable has not likely been virtualized. If allglobal variables pass this test, then detectVirt( ) returns true, andthe secure application may have some confidence that a virtualizationattack has not occurred. At this point, the program may takeapplication-specific measures, e.g., caching references to thenon-virtualized objects for subsequent use.

III. Flowchart Description

Features discussed herein are provided as example embodiments that maybe implemented in many different ways that may be understood by one ofskill in the art of data processing, without departing from the spiritof the discussion herein. Such features are to be construed only asexample embodiment features, and are not intended to be construed aslimiting to only those detailed descriptions.

FIG. 3 is a flowchart illustrating example operations of the system ofFIG. 1, according to example embodiments. In the example of FIG. 3 a, abaseline set of graph information associated with a predefined executionenvironment of a program may be obtained, the baseline set associatedwith a baseline time interval, the predefined execution environmentincluding predefined environment objects, at least a portion of thepredefined environment objects being dynamically mutatable (302). Forexample, the environment state determination component 104 may obtainthe baseline set 106 of graph information associated with the predefinedexecution environment 108 of the program 110, as discussed above.

A second set of graph information associated with the predefinedexecution environment may be obtained, the second set associated with asecond time interval that is later in time than the baseline timeinterval (304). For example, the environment state determinationcomponent 104 may obtain the second set 116 of graph information, asdiscussed above.

The baseline set and the second set may be compared to detectvirtualization activity associated with a dynamic runtime (306). Forexample, the environment state comparison component 140 may compare thebaseline set 106 and the second set 116, as discussed above.

For example, the predefined environment objects that are dynamicallymutatable may include one or more of environment variables, environmentfunctions, environment references, environment constant values, orenvironment state values (308).

For example, the predefined execution environment of the program mayinclude one or more of a predefined JAVASCRIPT execution environment ofa web browser, a predefined RUBY execution environment, or a predefinedPYTHON execution environment (310).

For example, the baseline set of graph information may include baselinestate information associated with predefined execution environmentobjects and relationships between the predefined execution environmentobjects, at the baseline time interval, wherein a graph associated withthe baseline set of graph information is induced based on graph nodesrepresenting the predefined execution environment objects and graphedges representing the relationships between the predefined executionenvironment objects (312), as indicated in FIG. 3 b.

For example, the second set of graph information may include secondstate information associated with the predefined execution environmentobjects and the relationships between the predefined executionenvironment objects, at the second time interval (314).

For example, the baseline set of graph information may be obtained basedon recursively first traversing nodes representing predefined executionenvironment objects that are reachable via references from globalvariables in the predefined execution environment, at the baseline timeinterval, and storing values associated with first traversal visitednodes in a first list (316). For example, the environment statedetermination component 104 may obtain the baseline set 106 of graphinformation based on recursively first traversing nodes representingpredefined execution environment objects 114 that are reachable viareferences from global variables in the predefined execution environment108, at the baseline time interval, and storing values associated withfirst traversal visited nodes in the first list 162, as discussed above.

For example, the second set of graph information may be obtained basedon recursively second traversing the nodes representing the predefinedexecution environment objects that are reachable via references fromglobal variables in the predefined execution environment, at the secondtime interval, and storing values associated with second traversalvisited nodes in a second list (318). For example, the environment statedetermination component 104 may obtain the second set 116 of graphinformation based on recursively second traversing the nodesrepresenting the predefined execution environment objects 114 that arereachable via references from global variables in the predefinedexecution environment 108, as discussed above.

For example, the baseline set and the second set may be compared basedon one or more of comparing a size of the first list with a size of thesecond list, or comparing a value of each element of the first list to avalue of a corresponding element of the second list (320). For example,the environment state comparison component 140 may compare the baselineset 106 and the second set 116 based on one or more of comparing a sizeof the first list 162 with a size of the second list 164, or comparing avalue of each element of the first list 162 to a value of acorresponding element of the second list 164, as discussed above.

For example, storage of the baseline set of graph information in aprivate storage location that is privately available to the predefinedexecution environment may be initiated, wherein the baseline set ofgraph information is available for use by the predefined executionenvironment, in lieu of at least a portion of the predefined environmentobjects that are dynamically mutatable (322), as indicated in FIG. 3 c.For example, the graph caching component 166 may initiate storage of thebaseline set 106 of graph information in a private storage location 168that is privately available to the predefined execution environment 110,as discussed above.

For example, validation of current output of at least one function thatis defined by the predefined execution environment may be initiated(324). For example, the environment function validation component 172may initiate the validation, as discussed above.

For example, the baseline set of graph information may be obtained viainvocation of the at least one function after validation of output(326). For example, the second set of graph information may be obtainedvia invocation of the at least one function after validation of output(328).

For example, the second set of graph information may be obtained viainvocation of a first non-virtualizable function, wherein the firstnon-virtualizable function is defined by the predefined executionenvironment (330), as indicated in FIG. 3 d.

For example, a third set of graph information associated with thepredefined execution environment may be obtained, at a third timeinterval that is later in time than the second time interval, the thirdset obtained via invocation of the first non-virtualizable function(332). For example, the environment state determination component 104may obtain the third set 170 of graph information associated with thepredefined execution environment 108, as discussed above.

For example, the baseline set and the third set may be compared todetect virtualization activity associated with the dynamic runtime(334). For example, the environment state comparison component 140 maycompare the baseline set 106 and the third set 170 to detectvirtualization activity associated with the dynamic runtime, asdiscussed above.

For example, a first supplemental set of graph information associatedwith a program graph that is induced based on predefined program objectsof the program, and predefined relationships between the predefinedprogram objects, may be obtained, at the baseline time interval (336),as indicated in FIG. 3 e. For example, the supplemental graph component150 may obtain the first supplemental set 152 of graph informationassociated with the program graph that is induced based on predefinedprogram objects of the program 110, and predefined relationships betweenthe predefined program objects, at the baseline time interval, asdiscussed above.

For example, a second supplemental set of graph information associatedwith the program graph may be obtained, at the second time interval(338). For example, the supplemental graph component 150 may obtain thesecond supplemental set 156 of graph information associated with theprogram graph, at the second time interval, as discussed above.

For example, the first supplemental set may be added to the baseline setof graph information via a second non-virtualizable function invoked bythe program, the second non-virtualizable function defined by thepredefined execution environment (340).

For example, the second supplemental set may be added to the second setof graph information via the second non-virtualizable function (342).

For example, the baseline set and the second set may be compared basedon comparing the baseline set that includes the first supplemental setand the second set that includes the second supplemental set, to detectthe virtualization activity associated with the dynamic runtime (344).For example, the environment state comparison component 140 may comparethe baseline set 106 and the second set 116 based on comparing thebaseline set 106 that includes the first supplemental set 152 and thesecond set 116 that includes the second supplemental set 156, to detectthe virtualization activity associated with the dynamic runtime, asdiscussed above.

FIG. 4 is a flowchart illustrating example operations of the system ofFIG. 1, according to example embodiments. In the example of FIG. 4 a, abaseline set of state information associated with predefined executionenvironment objects that are dynamically mutatable, and that areincluded in a predefined execution environment of a program, may beobtained, at least a portion of the predefined execution environmentobjects being dynamically mutatable, the baseline set of stateinformation indicating a baseline state of the predefined executionenvironment at a baseline time interval (402). For example, theenvironment state determination component 104 may obtain the baselineset 106, as discussed above.

A second set of state information associated with the predefinedexecution environment objects may be obtained, the second set of stateinformation indicating a second state of the predefined executionenvironment at a second time interval that is later in time than thebaseline time interval (404). For example, the environment statedetermination component 104 may obtain the second set 116, as discussedabove.

A comparison of the baseline set and the second set may be initiated todetermine virtualization activity associated with a dynamic runtime(406). For example, the environment state comparison component 140 maycompare the baseline set 106 and the second set 116, as discussed above.

For example, a validation of current output of at least one functionthat is defined by the predefined execution environment may be initiated(408). For example, the environment function validation component 172may initiate the validation, as discussed above.

For example, the baseline set of graph information may be obtained viainvocation of the at least one function after validation of output(410). For example, the second set of graph information may be obtainedvia invocation of the at least one function after validation of output(412), as discussed above.

For example, the baseline set of state information may be obtained basedon obtaining a baseline set of graph information associated with thepredefined execution environment of the program, wherein a graphassociated with the baseline set of graph information is induced basedon graph nodes representing the predefined execution environment objectsand graph edges representing the relationships between the predefinedexecution environment objects (414), as indicated in FIG. 4 b.

For example, the second set of state information may be obtained basedon obtaining a second set of graph information associated with thepredefined execution environment of the program, wherein a graphassociated with the second set of graph information is induced based onthe graph nodes representing the predefined execution environmentobjects and the graph edges representing the relationships between thepredefined execution environment objects, at the second time interval(416).

For example, the baseline set of graph information may include baselinestate information associated with the predefined execution environmentobjects and relationships between the predefined execution environmentobjects, at the baseline time interval, wherein a graph associated withthe baseline set of graph information is induced based on graph nodesrepresenting the predefined execution environment objects and graphedges representing the relationships between the predefined executionenvironment objects (418).

For example, the second set of graph information may include secondstate information associated with the predefined execution environmentobjects and the relationships between the predefined executionenvironment objects, at the second time interval (420).

For example, the second set of state information may be obtained viainvocation of a first non-virtualizable function, wherein the firstnon-virtualizable function is defined by the predefined executionenvironment (422), as indicated in FIG. 4 c

For example, the baseline set of graph information may be obtained basedon recursively first traversing nodes representing predefined executionenvironment objects that are reachable via references from globalvariables in the predefined execution environment, at the baseline timeinterval, and storing values associated with first traversal visitednodes in a first list (424).

For example, the second set of graph information may be obtained basedon recursively second traversing the nodes representing the predefinedexecution environment objects that are reachable via references fromglobal variables in the predefined execution environment, at the secondtime interval, and storing values associated with second traversalvisited nodes in a second list (426).

For example, the comparison of the baseline set and the second set maybe initiated based on one or more of comparing a size of the first listwith a size of the second list, or comparing a value of each element ofthe first list to a value of a corresponding element of the second list(428).

FIG. 5 is a flowchart illustrating example operations of the system ofFIG. 1, according to example embodiments. In the example of FIG. 5 a, abaseline set of graph information associated with a predefinedJAVASCRIPT execution environment of a web browser may be obtained, thebaseline set indicating a baseline state of the predefined JAVASCRIPTexecution environment at a baseline time interval (502). For example,the environment state determination component 104 may obtain thebaseline set 106 of graph information associated with the predefinedexecution environment 108 of the program 110, as discussed above.

A second set of graph information associated with the predefinedJAVASCRIPT execution environment may be obtained, the second setindicating a second state of the predefined JAVASCRIPT executionenvironment at a second time interval that is later in time than thebaseline time interval (504). For example, the environment statedetermination component 104 may obtain the second set 116 of graphinformation, as discussed above.

The baseline set and the second set may be compared to detectvirtualization activity associated with a dynamic runtime (506). Forexample, the environment state comparison component 140 may compare thebaseline set 106 and the second set 116, as discussed above.

For example, the second set of graph information may be obtained via afirst non-virtualizable function invoked by the web browser, the firstnon-virtualizable function defined by the predefined JAVASCRIPTexecution environment (508), as discussed above.

For example, a validation of current output of at least one functionthat is defined by the predefined JAVASCRIPT execution environment maybe initiated (510). For example, the environment function validationcomponent 172 may initiate the validation, as discussed above.

For example, the baseline set of graph information may be obtained viainvocation of the at least one function after validation of output(512), as discussed above. For example, the second set of graphinformation may be obtained via invocation of the at least one functionafter validation of output (514).

One skilled in the art of data processing will understand that there aremany ways of detecting virtualization in dynamic runtime environments,without departing from the spirit of the discussion herein.

Customer privacy and confidentiality have been ongoing considerations indata processing environments for many years. Thus, example techniquesfor virtualization detection may use user input and/or data provided byusers who have provided permission via one or more subscriptionagreements (e.g., “Terms of Service” (TOS) agreements) with associatedapplications or services associated with browsing. For example, usersmay provide consent to have their input/data transmitted and stored ondevices, though it may be explicitly indicated (e.g., via a useraccepted text agreement) that each party may control how transmissionand/or storage occurs, and what level or duration of storage may bemaintained, if any.

Implementations of the various techniques described herein may beimplemented in digital electronic circuitry, or in computer hardware,firmware, software, or in combinations of them (e.g., an apparatusconfigured to execute instructions to perform various functionality).

Implementations may be implemented as a computer program embodied in apure signal such as a pure propagated signal. Such implementations maybe referred to herein as implemented via a “computer-readabletransmission medium.”

Alternatively, implementations may be implemented as a computer programembodied in a machine usable or machine readable storage device (e.g., amagnetic or digital medium such as a Universal Serial Bus (USB) storagedevice, a tape, hard disk drive, compact disk, digital video disk (DVD),etc.), for execution by, or to control the operation of, data processingapparatus, e.g., a programmable processor, a computer, or multiplecomputers. Such implementations may be referred to herein as implementedvia a “computer-readable storage medium” or a “computer-readable storagedevice” and are thus different from implementations that are purelysignals such as pure propagated signals.

A computer program, such as the computer program(s) described above, canbe written in any form of programming language, including compiled,interpreted, or machine languages, and can be deployed in any form,including as a stand-alone program or as a module, component,subroutine, or other unit suitable for use in a computing environment.The computer program may be tangibly embodied as executable code (e.g.,executable instructions) on a machine usable or machine readable storagedevice (e.g., a computer-readable storage medium). A computer programthat might implement the techniques discussed above may be deployed tobe executed on one computer or on multiple computers at one site ordistributed across multiple sites and interconnected by a communicationnetwork.

Method steps may be performed by one or more programmable processorsexecuting a computer program to perform functions by operating on inputdata and generating output. The one or more programmable processors mayexecute instructions in parallel, and/or may be arranged in adistributed configuration for distributed processing. Examplefunctionality discussed herein may also be performed by, and anapparatus may be implemented, at least in part, as one or more hardwarelogic components. For example, and without limitation, illustrativetypes of hardware logic components that may be used may includeField-programmable Gate Arrays (FPGAs), Program-specific IntegratedCircuits (ASICs), Program-specific Standard Products (ASSPs),System-on-a-chip systems (SOCs), Complex Programmable Logic Devices(CPLDs), etc.

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. Elements of a computer may include atleast one processor for executing instructions and one or more memorydevices for storing instructions and data. Generally, a computer alsomay include, or be operatively coupled to receive data from or transferdata to, or both, one or more mass storage devices for storing data,e.g., magnetic, magneto optical disks, or optical disks. Informationcarriers suitable for embodying computer program instructions and datainclude all forms of nonvolatile memory, including by way of examplesemiconductor memory devices, e.g., EPROM, EEPROM, and flash memorydevices; magnetic disks, e.g., internal hard disks or removable disks;magneto optical disks; and CD ROM and DVD-ROM disks. The processor andthe memory may be supplemented by, or incorporated in special purposelogic circuitry.

To provide for interaction with a user, implementations may beimplemented on a computer having a display device, e.g., a cathode raytube (CRT), liquid crystal display (LCD), or plasma monitor, fordisplaying information to the user and a keyboard and a pointing device,e.g., a mouse or a trackball, by which the user can provide input to thecomputer. Other kinds of devices can be used to provide for interactionwith a user as well; for example, feedback provided to the user can beany form of sensory feedback, e.g., visual feedback, auditory feedback,or tactile feedback. For example, output may be provided via any form ofsensory output, including (but not limited to) visual output (e.g.,visual gestures, video output), audio output (e.g., voice, devicesounds), tactile output (e.g., touch, device movement), temperature,odor, etc.

Further, input from the user can be received in any form, includingacoustic, speech, or tactile input. For example, input may be receivedfrom the user via any form of sensory input, including (but not limitedto) visual input (e.g., gestures, video input), audio input (e.g.,voice, device sounds), tactile input (e.g., touch, device movement),temperature, odor, etc.

Further, a natural user interface (NUI) may be used to interface with auser. In this context, a “NUI” may refer to any interface technologythat enables a user to interact with a device in a “natural” manner,free from artificial constraints imposed by input devices such as mice,keyboards, remote controls, and the like.

Examples of NUI techniques may include those relying on speechrecognition, touch and stylus recognition, gesture recognition both on ascreen and adjacent to the screen, air gestures, head and eye tracking,voice and speech, vision, touch, gestures, and machine intelligence.Example NUI technologies may include, but are not limited to, touchsensitive displays, voice and speech recognition, intention and goalunderstanding, motion gesture detection using depth cameras (e.g.,stereoscopic camera systems, infrared camera systems, RGB (red, green,blue) camera systems and combinations of these), motion gesturedetection using accelerometers/gyroscopes, facial recognition, 3Ddisplays, head, eye, and gaze tracking, immersive augmented reality andvirtual reality systems, all of which may provide a more naturalinterface, and technologies for sensing brain activity using electricfield sensing electrodes (e.g., electroencephalography (EEG) and relatedtechniques).

Implementations may be implemented in a computing system that includes aback end component, e.g., as a data server, or that includes amiddleware component, e.g., an application server, or that includes afront end component, e.g., a client computer having a graphical userinterface or a Web browser through which a user can interact with animplementation, or any combination of such back end, middleware, orfront end components. Components may be interconnected by any form ormedium of digital data communication, e.g., a communication network.Examples of communication networks include a local area network (LAN)and a wide area network (WAN), e.g., the Internet.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims. While certain features of thedescribed implementations have been illustrated as described herein,many modifications, substitutions, changes and equivalents will nowoccur to those skilled in the art. It is, therefore, to be understoodthat the appended claims are intended to cover all such modificationsand changes as fall within the scope of the embodiments.

What is claimed is:
 1. A system comprising: a device that includes atleast one processor, the device including a virtualization detectionengine comprising instructions tangibly embodied on a computer readablestorage medium for execution by the at least one processor, thevirtualization detection engine including: an environment statedetermination component configured to: obtain a baseline set of graphinformation associated with a predefined execution environment of aprogram, the baseline set associated with a baseline time interval, thepredefined execution environment including predefined environmentobjects, at least a portion of the predefined environment objects beingdynamically mutatable, and obtain a second set of graph informationassociated with the predefined execution environment, the second setassociated with a second time interval that is later in time than thebaseline time interval; and an environment state comparison componentconfigured to compare the baseline set and the second set to detectvirtualization activity associated with a dynamic runtime.
 2. The systemof claim 1, wherein: the predefined environment objects that aredynamically mutatable include one or more of: environment variables,environment functions, environment references, environment constantvalues, or environment state values.
 3. The system of claim 1, wherein:the predefined execution environment of the program includes one or moreof: a predefined JAVASCRIPT execution environment of a web browser, apredefined RUBY execution environment, or a predefined PYTHON executionenvironment.
 4. The system of claim 1, wherein: the second set of graphinformation is obtained via invocation of a first non-virtualizablefunction, wherein the first non-virtualizable function is defined by thepredefined execution environment.
 5. The system of claim 4, furthercomprising: a supplemental graph component configured to: obtain a firstsupplemental set of graph information associated with a program graphthat is induced based on predefined program objects of the program, andpredefined relationships between the predefined program objects, at thebaseline time interval, obtain a second supplemental set of graphinformation associated with the program graph, at the second timeinterval, add the first supplemental set to the baseline set of graphinformation via a second non-virtualizable function invoked by theprogram, the second non-virtualizable function defined by the predefinedexecution environment, and add the second supplemental set to the secondset of graph information via the second non-virtualizable function,wherein: the environment state comparison component is configured tocompare the baseline set and the second set based on comparing thebaseline set that includes the first supplemental set and the second setthat includes the second supplemental set, to detect the virtualizationactivity associated with the dynamic runtime.
 6. The system of claim 4,wherein: the environment state determination component is configured toobtain a third set of graph information associated with the predefinedexecution environment, at a third time interval that is later in timethan the second time interval, the third set obtained via invocation ofthe first non-virtualizable function, wherein: the environment statecomparison component is configured to compare the baseline set and thethird set to detect the virtualization activity associated with thedynamic runtime.
 7. The system of claim 1, wherein: the baseline set ofgraph information includes baseline state information associated withthe predefined execution environment objects and relationships betweenthe predefined execution environment objects, at the baseline timeinterval, wherein a graph associated with the baseline set of graphinformation is induced based on graph nodes representing the predefinedexecution environment objects and graph edges representing therelationships between the predefined execution environment objects, andthe second set of graph information includes second state informationassociated with the predefined execution environment objects and therelationships between the predefined execution environment objects, atthe second time interval.
 8. The system of claim 1, wherein: theenvironment state determination component is configured to obtain thebaseline set of graph information based on recursively first traversingnodes representing predefined execution environment objects that arereachable via references from global variables in the predefinedexecution environment, at the baseline time interval, and storing valuesassociated with first traversal visited nodes in a first list, and theenvironment state determination component is configured to obtain thesecond set of graph information based on recursively second traversingthe nodes representing the predefined execution environment objects thatare reachable via references from global variables in the predefinedexecution environment, at the second time interval, and storing valuesassociated with second traversal visited nodes in a second list.
 9. Thesystem of claim 8, wherein: the environment state comparison componentis configured to compare the baseline set and the second set based onone or more of: comparing a size of the first list with a size of thesecond list, or comparing a value of each element of the first list to avalue of a corresponding element of the second list.
 10. The system ofclaim 1, further comprising: a graph caching component configured toinitiate storage of the baseline set of graph information in a privatestorage location that is privately available to the predefined executionenvironment, wherein the baseline set of graph information is availablefor use by the predefined execution environment, in lieu of at least aportion of the predefined environment objects that are dynamicallymutatable.
 11. The system of claim 1, further comprising: an environmentfunction validation component configured to initiate validation ofcurrent output of at least one function that is defined by thepredefined execution environment, wherein the baseline set of graphinformation is obtained via invocation of the at least one functionafter validation of output, wherein the second set of graph informationis obtained via invocation of the at least one function after validationof output.
 12. A method comprising: obtaining a baseline set of graphinformation associated with a predefined JAVASCRIPT executionenvironment of a web browser, the baseline set indicating a baselinestate of the predefined JAVASCRIPT execution environment at a baselinetime interval; obtaining a second set of graph information associatedwith the predefined JAVASCRIPT execution environment, the second setindicating a second state of the predefined JAVASCRIPT executionenvironment at a second time interval that is later in time than thebaseline time interval; and comparing the baseline set and the secondset to detect virtualization activity associated with a dynamic runtime.13. The method of claim 12, wherein: the second set of graph informationis obtained via a first non-virtualizable function invoked by the webbrowser, the first non-virtualizable function defined by the predefinedJAVASCRIPT execution environment.
 14. The method of claim 12, furthercomprising: initiating a validation of current output of at least onefunction that is defined by the predefined JAVASCRIPT executionenvironment, wherein the baseline set of graph information is obtainedvia invocation of the at least one function after validation of output,wherein the second set of graph information is obtained via invocationof the at least one function after validation of output.
 15. A computerprogram product tangibly embodied on a computer-readable storage mediumand including executable code that causes at least one data processingapparatus to: obtain, via a device processor, a baseline set of stateinformation associated with predefined execution environment objectsthat are included in a predefined execution environment of a program, atleast a portion of the predefined execution environment objects beingdynamically mutatable, the baseline set of state information indicatinga baseline state of the predefined execution environment at a baselinetime interval; obtain a second set of state information associated withthe predefined execution environment objects, the second set of stateinformation indicating a second state of the predefined executionenvironment at a second time interval that is later in time than thebaseline time interval; and initiate a comparison of the baseline set ofstate information and the second set of state information to determinevirtualization activity associated with a dynamic runtime.
 16. Thecomputer program product of claim 15, wherein the executable code isconfigured to cause the at least one data processing apparatus to:obtain the baseline set of state information based on obtaining abaseline set of graph information associated with the predefinedexecution environment of the program, wherein a graph associated withthe baseline set of graph information is induced based on graph nodesrepresenting the predefined execution environment objects and graphedges representing the relationships between the predefined executionenvironment objects, and obtain the second set of state informationbased on obtaining a second set of graph information associated with thepredefined execution environment of the program, wherein a graphassociated with the second set of graph information is induced based onthe graph nodes representing the predefined execution environmentobjects and the graph edges representing the relationships between thepredefined execution environment objects, at the second time interval.17. The computer program product of claim 16, wherein: the baseline setof graph information includes baseline state information associated withthe predefined execution environment objects and relationships betweenthe predefined execution environment objects, at the baseline timeinterval, wherein a graph associated with the baseline set of graphinformation is induced based on graph nodes representing the predefinedexecution environment objects and graph edges representing therelationships between the predefined execution environment objects, andthe second set of graph information includes second state informationassociated with the predefined execution environment objects and therelationships between the predefined execution environment objects, atthe second time interval.
 18. The computer program product of claim 16,wherein the executable code is configured to cause the at least one dataprocessing apparatus to: obtain the second set of state information viainvocation of a first non-virtualizable function, wherein the firstnon-virtualizable function is defined by the predefined executionenvironment; obtain the baseline set of graph information based onrecursively first traversing nodes representing predefined executionenvironment objects that are reachable via references from globalvariables in the predefined execution environment, at the baseline timeinterval, and storing values associated with first traversal visitednodes in a first list, and obtain the second set of graph informationbased on recursively second traversing the nodes representing thepredefined execution environment objects that are reachable viareferences from global variables in the predefined executionenvironment, at the second time interval, and storing values associatedwith second traversal visited nodes in a second list.
 19. The computerprogram product of claim 18, wherein the executable code is configuredto cause the at least one data processing apparatus to: initiate thecomparison of the baseline set of state information and the second setof state information based on one or more of: comparing a size of thefirst list with a size of the second list, or comparing a value of eachelement of the first list to a value of a corresponding element of thesecond list.
 20. The computer program product of claim 15, wherein theexecutable code is configured to cause the at least one data processingapparatus to: initiate a validation of current output of at least onefunction that is defined by the predefined execution environment,wherein the baseline set of graph information is obtained via invocationof the at least one function after validation of output, wherein thesecond set of graph information is obtained via invocation of the atleast one function after validation of output.